Evaluating synthetic speech in an Irish CALL application: influences of predisposition and of the holistic environment

This paper reports an evaluation of Irish synthetic voices in the context of a virtual reality CALL application. In addition to eliciting subjects’ ratings of the synthetic voices, the evaluation focusses particularly on (1) the extent to which prior attitudes to synthetic voices affected users' satisfaction ratings and (2) the extent to which reactions to the synthetic voices were influenced by users' engagement with the other (non-speech) dimensions of the CALL application. The particular application, Fáilte go TCD, was developed specifically for this purpose and uses virtual reality scenes where the animated characters converse in Irish (synthetic voices). Evaluations were carried out using Likert scale-based questionnaires. Results showed broadly positive ratings of the Irish synthetic voices in terms of intelligibility, quality and naturalness. They further indicate that (1) users' prior attitudes towards synthetic speech had a major influence on their reaction to the CALL application and (2) satisfaction levels with the synthetic voices are highly correlated with the rating accorded to other, non-speech dimensions of the platform, suggesting that the different aspects are judged in a holistic way.